144: Load-simulation harness: in-cluster load generation with SLO gates and evidence#166
Conversation
…force-factory-81a11a73
…force-factory-81a11a73
# Conflicts: # src/fleet/internal-fleet-client.test.ts
|
@coderabbitai review |
✅ Action performedReview finished.
|
|
Warning Review limit reached
Next review available in: 29 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (10)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
@cubic-dev-ai ultrareview: focus on Kubernetes context isolation, cleanup, untrusted profile/resource inputs, k6 evidence correctness, and CI reproducibility. |
@kjgbot Couldn't start the ultrareview: please upgrade to a paid plan to review bot-authored PRs. |
Part of Epic: Factory Verification & Environments (#141). Depends on the EnvironmentProvider (#142) and stack deployer (#143).
Why
"Fully tested end to end" includes behavior under load, not just a single happy-path request. The Factory needs to drive realistic traffic at a deployed stack and decide pass/fail against SLOs — otherwise a feature that works for one request but collapses at concurrency ships green.
What
runLoad(env, profile): LoadResultsurface fromsrc/index.ts.Acceptance
src/index.ts.✅ End-to-end verification — Factory success criteria (REQUIRED)
"Compiles" is not done. Fire real load at a real deployed service and prove the SLO gate flips correctly.
Run it
The E2E must:
Deliverable
test/e2e/load-harness.e2e.tsrunning a real load run (Container/CI k6) against a deployed Cloudflare environment with both a passing and a failing SLO assertion, one command in CI.Anchor files
src/environments/load-harness.ts(+ k6 Container/CI runner)src/ports/environment.ts/src/environments/*(from EnvironmentProvider port + Cloudflare ephemeral-environment provider (Workers for Platforms + Containers); K8s/Daytona optional #142/Verification-stack descriptor contract + deployer: repos declare their full stack, Factory brings it up #143)src/observability/*(evidence shape precedent),src/index.tsFixes #144
Summary by cubic
Adds an in-cluster load harness that runs k6 as a Kubernetes Job, checks SLOs, and emits machine-readable evidence. Hardened to fail closed on zero-traffic and validate evidence integrity; this gates deployed stacks and fulfills #144 in the Factory Verification & Environments epic.
New Features
runLoad(environment, profile)creates a k6 Job, parses results, applies SLOs, and returns pass/fail with measurements and evidence JSON (factory.load.evidence.v1).loadLoadProfile(path)): targets (endpoint or URL), VU/RPS with optional ramp, thresholds, and latency histogram; arrival-rate and VU scenarios supported.requestCountviolation.KubectlLoadJobClient, hardened Job/ConfigMap with pinnedDEFAULT_K6_IMAGE, andrenderK6Script.test/e2e/load-harness.e2e.tsdeploys a sample service, runs pass/fail SLOs, writesartifacts/load-e2e/*.json; GitHub Actions job creates a kind cluster, pins the kubectl context for the run, and uploads evidence artifacts bound to the head SHA.docs/load-harness.mdwith usage and profile examples.Dependencies
tsxandtest:e2e:loadscript.Written for commit 94d2454. Summary will update on new commits.